#Transformer efficiency

Researchers from Edinburgh, Cohere, and Meta demonstrate that large sparse models can outperform smaller dense models for long-context LLMs by leveraging sparse attention, offering new scaling laws and standardized methods.